Photo collection representation based on viewpoint clustering

The users of digital cameras often take multiple photographs of the same scene. Such multiple shots usually have a special meaning to the photographer, and require further actions, e.g. selection of the best exposure/composition/portrait or stitching several images into a panorama or composite image. We present a method of fast retrieval of all groups of shots taken from the same viewpoint. This task is different from the recently emerged near-duplicate detection problem because, in our case, the multiple shots differ not only by photometric and simple geometric transformations; they can have a little or no overlap, and large variations of objects may be presented. Therefore, we solve a general multiple image registration problem by extracting local image descriptors, their matching, and recovering geometric transformation between images. Initially, the photo-collection is divided in time-based clusters, which are then refined by extracting connected components from the global image registration graph. The method has been applied to real consumer photo-collections, and we show that depending on individual camera usage styles, user collections contain from 15% to 90% of photos requiring further attention. The presented system automates the otherwise manual work of selecting a series of similar images.